Hugging Face
VERIFIED@hugging-faceOpen-source AI hub. Ships Transformers, Text Generation Inference, Datasets, Tokenizers, Smolagents — the libraries running half the open ML world.
Skills
7Text Embeddings Inference — High-Performance Embedding Server by Hugging Face
A blazing-fast inference server for text embedding and reranking models. TEI serves any Sentence Transformers or cross-encoder model with optimized Rust and CUDA kernels, token-based dynamic batching, and an OpenAI-compatible API.
Hugging Face Tokenizers — Fast Text Tokenization for ML Pipelines
Hugging Face Tokenizers is a Rust-powered tokenization library with Python bindings that implements BPE, WordPiece, Unigram, and SentencePiece tokenizers with training and encoding speeds of gigabytes per second, used as the backbone for Transformers model tokenization.
Hugging Face Datasets — Access and Process ML Datasets at Scale
Hugging Face Datasets is a Python library for efficiently loading, processing, and sharing machine learning datasets with Apache Arrow-backed memory mapping, streaming support, and access to thousands of community datasets on the Hub.
Text Generation Inference (TGI) — Hugging Face Production LLM Server
TGI is Hugging Face's production-grade LLM inference server. It powers HF Inference Endpoints with continuous batching, tensor parallelism, quantization, and OpenAI-compatible APIs — handling thousands of requests per second.
Hugging Face Transformers — The Universal Library for Pretrained Models
transformers is the de-facto Python library for using and fine-tuning pretrained models — BERT, GPT, Llama, Whisper, ViT, and 250,000+ others. One unified API works across PyTorch, TensorFlow, and JAX.
Smolagents — Lightweight Agent Framework by HuggingFace
Minimalist Python agent framework by HuggingFace. Build agents with tool use in under 30 lines of code. Supports code-based agents that write and execute Python instead of JSON tool calls. 15,000+ stars.